On-Device Deep Learning Inference for System-on-Chip (SoC) Architectures

As machine learning becomes ubiquitous, the need to deploy models on real-time, embedded systems will become increasingly critical. This is especially true for deep solutions, whose large pose interesting challenges target architectures at “edge” that are resource-constrained. The realization of learning, and being driven by availability specialized hardware, such as system-on-chip which provide some alleviation constraints. Equally important, however, operating run this specifically ability leverage commercial real-time which, unlike general purpose Linux, can low-latency, deterministic execution required embedded, potentially safety-critical, applications edge. Despite this, studies considering integration systems, learning/deep algorithms remain limited. In particular, better mechanisms scheduling in context prove be critical these technologies move order address challenges, we present a resource management framework designed dynamic on-device approach allocation limited resources processing environment. These types necessary support behavior control components contained edge nodes. To validate effectiveness our approach, applied rigorous schedulability analysis set randomly generated simulated task sets then verified most time applications, tasks maintained low-latency even during off-nominal conditions. practicality was demonstrated integrating it into system (VxWorks) running typical image application perform simple object detection. results indicate proposed leveraged facilitate with platforms, including widely-used, industry-standard systems.